Field of the Invention
The present invention generally relates to virtual keyboards, and more particularly, to a virtual keyboard text entry system for devices with touch sensitive input panels or touch sensitive display screens.
Description of Related Art
The Standard Mechanical Keyboard, an example of which is depicted in FIG. 1, uses mechanical switches that are tapped to produce electronic outputs corresponding to individual characters. This keyboard system continues to be popular, and one of the fastest, tools for textual input into a device. However, the recent commercial popularity of handheld computing devices with touch sensitive technology has prompted the adaptation of the Standard Mechanical Keyboard, into a Standard Virtual Keyboard that retains many properties of its predecessor.
More specifically, a Standard Virtual Keyboard, examples of which are shown in FIG. 2, is a system and method used for entering text character-by-character using a keyboard displayed or printed on a touch-sensitive input panel or display screen, where contact by a user finger or fingers with the surface of the display generates input signals to the system corresponding to the location of contact. The Standard Virtual Keyboard divides a reserved entry area into discrete individual regions, or keys, that are struck individually, in a tapping motion called a keystroke, in order to output a particular character associated with that key. Standard Virtual Keyboards typically preserve many properties of Standard Mechanical Keyboards, including their overall rectangular shape, the organization of rectangular-shaped keys in a grid, and significant fragments of the original key layout. While numerous variants of this system exist, the most popular of these systems all use the basic features described above, justifying the depiction of the Standard Virtual Keyboard as “standard”.
For example, a standard layout for a Standard Mechanical Keyboard refers to (1) the geometric properties of the Standard Mechanical Keyboard's keys, including the size of each key and the division of the keyboard into rows having a certain number of keys. The standard layout also encompasses (2) the assignments of characters to the various keys (i.e., the QWERTY system), including the positional relationships between these characters (e.g., a “k” is directly to the left of an “l”). Variations of the standard layout are conventionally described in terms of being “similar to” a standard layout with respect to these properties.
In various countries, different character sets (see, e.g., www.unicode.org) are used, and therefore standard mappings of characters to keys on the Standard Mechanical Keyboard have been devised for those character sets. Therefore, standard layouts exist for Dutch, Japanese, Italian, etc. However, if some new layout L (composed of both key geometry and character assignments) is similar to the standard layout for English (i.e., the QWERTY layout), then it may be straightforward to adapt L for some new character set C so that the same similarities exist with the corresponding standard layout for C. This is because all of these standard layouts use the same underlying keyboard and key geometry, differing only in which characters are mapped to the keys. Therefore, L could be adapted for C simply by a process of character remapping.
A standard layout for a given character set is significant for three reasons. First, users of that character set have already taken significant time and effort to learn to type using that standard layout. Therefore, to the degree that a novel keyboard uses a layout that is similar to the standard layout, it will be easier for these users to learn the new layout.
Second, users of a standard layout have obtained some degree of “muscle memory” of that layout, where muscle memory is the ability of the human neuromuscular system to “remember” the locations of individual keys and quickly perform the micro-movements needed to position the hand near the key and then to strike the key with a finger. Muscle memory allows tasks to be performed more quickly and accurately, so its acquisition is a significant benefit. Therefore, to the degree that a novel keyboard layout is similar to a standard layout, muscle memory of the standard layout can be accessed and used when typing with the novel keyboard, leading to speed and accuracy benefits.
Finally, familiarity and muscle memory help greatly reduce the cognitive effort required to type successfully. Therefore, to the degree that a novel keyboard layout is similar to a standard layout, the novel keyboard will require less cognitive effort of users. All the above advantages of similarity to a standard layout can increase user acceptance of a novel layout.
The transition of the Standard Mechanical Keyboard to virtually has exposed limitations of the system, which was in use long before computers and touch screens were conceived of. For example, for effective use, a Standard Mechanical Keyboard: (1) should be at least a certain size; (2) should be used in a somewhat constrained user posture and spatial attitude to the keyboard; and (3) should be typed upon with two hands. However, handheld touch-screen computing devices, the conventional field of use for the Standard Virtual Keyboard, typically place stress on the above constraints, as they are usually small, are used in a variety of postures and attitudes, and are often typed upon using only one hand. More specifically, the geometry and use patterns of these new devices significantly stress and degrade the original design goals of the standard mechanical keyboard, even after significant adaptation for the virtual environment.
There are at least two commercially dominant classes of handheld touch-screen computers in use today, two of which are shown in FIG. 2: smartphones 10 and tablets 20. Both of these device types are significantly narrower than a Standard Mechanical Keyboard: popular exemplars of these devices are 9.5″ wide for a tablet, 2.5″ wide for a smartphone, and 12″ wide for a mechanical laptop keyboard. In order to adapt to these smaller widths, the Standard Virtual Keyboard eliminates many keys and their associated characters from the primary input screen, including the row of function keys (e.g., F1) and the navigation keys (e.g., arrow keys, Home, etc.). The primary input screen, shown on both the devices in FIG. 2, contains the alphabetic keys and keys for a small number of other characters that are very frequently typed, such as the Space key. In total, these screens contain only about 35 keys, while a Standard Mechanical Keyboard typically contains over 80 keys. Other characters and their keys (e.g., numerical keys) are removed to a secondary input screen (not shown) that is similar to the primary input screen in organization, but outputs different characters. This secondary input screen is typically invoked using a dedicated key 30, 40. Additional keys located on the primary or secondary input screens can be used to invoke additional input screens. Combined, all the input screens can be used to type all the keys typically found in the Standard Mechanical Keyboard.
The Standard Virtual Keyboard is a partial success, in that two-handed typing can be performed on at least the larger tablets. However, the postural requirements for successful two-handed typing are somewhat difficult. A user typically either balances a tablet on their lap, which is often precarious and uncomfortable, or sits at a table supporting the tablet, which introduces ergonomic problems experienced even by users of mechanical keyboards, but potentially more serious for virtual keyboard users because of table height variations. Additionally, because one of the attractions of tablets is their quick, convenient use in real-world postures (e.g., curled up on the sofa), many users prefer not to adopt the constrained postures described above.
Subtle differences from the Standard Mechanical Keyboard typing experience also make the Standard Virtual Keyboard experience less satisfactory. For example, the virtual keyboard and the display share the device surface. The keyboard therefore uses a large part of the overall device surface, reducing available display space. The sharing of the device surface between keyboard and display also degrades typing efficiency. Typically, to better see the display, a user prefers to tilt the device to some angle between horizontal and vertical. However, this tilt interferes with effective typing on a standard keyboard, which requires the key surface to be horizontal so that the hands can quickly and easily move over the keyboard at a set distance above it.
Even disregarding the above issues, the narrowness of a tablet requires a two-handed typist to maintain both hands in the palms-down typing position, but closer together than is the case with mechanical keyboards. This posture may be tiring to maintain over time, and may degrade typing speed. A final difficulty is that because a touch surface registers all finger touches as potential key strokes, between key strokes a user must maintain both hands at a small height above the surface, in contrast to mechanical typing, where the fingers may rest lightly on the keyboard. This floating position can be fatiguing for the user over time.
Another difficulty with the standard virtual keyboard concerns the use of multiple keyboard input screens to type the full set of characters used by the keyboard. The main disadvantage of this multi-screen solution is that typing speed suffers. For example, when a secondary input screen is invoked, the entire touch surface must be redrawn. The redrawing process is computationally costly and often imposes a slight delay that is perceptible to the user. This redrawing process must be repeated when the user returns to the primary input screen. Another reason typing speed suffers is the relatively complex sequence of keystrokes required to type a key residing on a secondary input screen. For example, to type the colon character, “:”, on the keyboard in FIG. 2, a user must, in sequence, (1) tap the ‘123’ key on the primary input screen in order to invoke the secondary input screen; (2) tap the colon key on the secondary input screen; and, in preparation for further typing on the primary input screen, (3) tap the ‘123’ key on the secondary input screen in order to return to the primary input screen. Therefore, typing a colon typically requires a sequence of three keystrokes, which typically takes three times as long to type and also increases the chance of a typing error. A final disadvantage of using multiple input screens is that rapidly switching between multiple screens can be disorienting to the user, especially if the key layouts substantially differ between the screens.
For many of the above reasons, but particularly to avoid the inconvenience and discomfort of adopting a restrictive two-handed posture, many users prefer to type one-handed on tablets. However, one-handed typing on any tablet's Standard Virtual Keyboard can be extremely inefficient. One might expect one-handed typing in this context to be about half as fast as two-handed typing, but it may actually be even slower. This is because typically a Standard Keyboard (either Virtual or Mechanical) is designed for two-handed typing, and therefore places all keys within a short distance of one of the expected ten fingers. Therefore, a one-handed typist often has to move their hand half the width of the keyboard to reach a letter, dramatically reducing the typing rate.
Furthermore, a one-handed typist can use only the muscle memories of one hand, which concern keys on only (roughly) half the keyboard. Therefore, striking keys on the other half of the keyboard may especially be slow. Even the remaining muscle memory is significantly disrupted, because the one typing hand is continually moving long distances from side to side. For many users, one-handed typing devolves to one-fingered typing, in a “hunt and peck” experience in which each key is painstakingly searched for.
Typing using standard virtual keyboards on smartphones is, if anything, even less efficient and convenient because of their extremely small size. Conventional two-handed typing, in which the hands hover over the keyboard, is near impossible when the phone is placed in a portrait orientation because the screen is typically too narrow to admit the use of both hands. Conventional two-handed typing is barely possible on a smartphone placed in landscape orientation, but the user must adopt an especially uncomfortable posture and the hands obscure the typing surface. The typing surface itself consumes most of the device area, leaving almost no space for the display. In addition, the individual keys are very small, requiring dramatically more accuracy to successfully strike the desired key. This need for extreme accuracy disrupts a fundamental speed advantage of standard keyboards: multi-finger typing, or, the ability of all five fingers to strike keys in quick succession, or even simultaneously. The multi-finger typing style, in which muscle memory plays a large role, is much less workable on a smartphone, because the user makes so many more mistakes at full speed when the keys are so small. For all these reasons, conventional two-handed typing on smartphones is infeasible.
In response, many users have adopted a new form of typing using smartphone keyboards comprising a two-thumb typing method, shown in FIG. 3, in which the phone is held up in both hands and both thumbs are used to press keys. Two-thumb typing achieves a fair degree of speed and accuracy since the two thumbs are more dexterous than the fingers and strike in quick succession, but the method has two disadvantages. First, the phone must be held up with both hands, which eventually becomes tiring, and makes the hands unavailable for other tasks. Second, two-thumb typing is not really feasible on tablets, both because the keyboard, which spans the width of the device, is too large for the thumbs to reach all keys, and because tablets may be too heavy to hold up for extended periods. Therefore, this typing method is not generalizable. In other words, a user who invests the time in developing two-thumb muscle memory on a smartphone must find some other method of typing for a tablet.
Another typing method for use with smartphone standard virtual keyboards is to type one-handed. This method performs better on smartphones than on tablets, because the hand needs to move a shorter distance to position a finger to strike any key. However, one-handed, multi-finger typing on a smartphone is still too error-prone to be practical, because the keys are too small. Instead, users employ one-fingered typing, as shown in FIG. 4, or one-thumb typing, so that they need only concentrate on one finger striking the correct keys. Using only one digit to strike keys obviously drastically limits the attainable speed, and typing accuracy is challenged even with one finger or thumb because of the small key size. To support one-digit typing, the hand often moves almost two inches laterally, making it difficult to develop effective muscle memory, which relies on smaller-scale movements. Fundamentally, striking a succession of small rectangles, each an inch or more away from the last, is a difficult task requiring a great deal of practice and concentration to master. A final important disadvantage is that, one-fingered typing, like two-thumb typing, is not generalizable, because the size differential between smartphone and tablet keyboards is so great as to make the one-fingered typing experience on these devices very different. Therefore, a user mastering one-fingered typing on a smartphone cannot transfer this mastery to tablet use, and vice versa.
A plethora of virtual keyboard systems have been developed to correct the speed and usability shortcomings of the Standard Virtual Keyboard. These systems may be categorized by initially enumerating a set of properties that any virtual keyboard used for tablet and/or smartphone typing should have to be successful, then the various keyboard systems may be categorized according to which properties each possesses.
In the inventor's opinion, the following properties should be implemented to make a successful virtual keyboard system.
A first property (1) of a successful virtual keyboard system is that they should support multi-finger typing with only one hand. While the foregoing provides reasons for this requirement, the most compelling are that: (a) on a tablet, two-handed typing in relaxed postures is difficult; and (b) on a smartphone, two-handed typing in any posture is very difficult. Since one-handed typing is most favorable, a successful system should organize its keys to support such a typing style, primarily by including most or all of the most frequently typed characters from the Standard Layout within easy reach of one hand. Some methods of accomplishing this might include making the keys smaller, and/or changing their relative positions, and/or changing the shapes of the keys.
A second property (2) of a successful virtual keyboard system is that they should somehow maintain the typing speeds associated with the Standard Mechanical Keyboard, despite the changes to the layout made in order to achieve multi-finger typing with only one hand. The most likely way this might be achieved is by organizing the keys in some fashion around the fingers, so that smaller and more uniform movements would be sufficient for fingers to reach and strike the keys. Such an organization may boost speed, because smaller movements are made more quickly than large movements. Additionally, a system consisting largely of smaller movements may be easier to type, because it may be difficult for fingers to alternate between making small and large movements. This could boost typing accuracy, which may in turn further boost speed, because the user may need to waste less time correcting mistakes.
In the inventor's opinion, the following beneficial ways for organizing keys around the fingers could be implemented to make a successful virtual keyboard system. Specific beneficial ways that the keys could be organized around the fingers may include the following. First (a), the field of keys typed by one hand could be arranged so that each key's angle matches the angle made by the hand to the touch surface edge: this angle is often approximately 30-45 degrees. Second (b), the field of keys typed by one hand could be arranged so that it more closely matches the overall shape of the hand. Third (c), the principle of (b) might be extended by dividing the keys into groups so that each group is placed near an individual finger which is responsible for typing that group. Organizing keys in this fashion allows each key to be struck using finger movement alone, without moving the arm, promoting typing accuracy. Fourth (d), the principle of (c) might be extended by arranging the group of keys associated with one finger in a columnar fashion, so that the keys are placed under the area occupied by a finger, in the area that is easily reached by the finger in its natural range of motion. This arrangement may offer the greatest scope for minimizing and regularizing the movements required to type any arbitrary key. Ideally, such an arrangement could place keys in an area roughly matching the width of individual user fingers so that the side-to-side distance required to reach keys may be minimized.
A third property (3) of a successful virtual keyboard system is that they should be at least partially similar to the Standard Layout, so that they would be easier for the typical user to learn.
A fourth property (4) of a successful virtual keyboard system is that they should comprise an interactive process that allows the layout to be fit to the user's hand. Supporting this kind of close, customized fit should further reduce the distances required to type individual keys, further boosting typing accuracy and speed.
Many virtual keyboard systems explore property (2a) by splitting the Standard Keyboard into left and right halves and then changing the angle of each half to better match that of its associated hand. This is type of virtual keyboard system is described in U.S. Patent Application Publication No. 2012/0144337, which also implements a form of (2b) by adjusting the width of individual keys to match a user's hand width, in an interactive process.
U.S. Patent Application Publication No. 2010/0259561, also splits and angles the keyboard, and also implements a form of (2b), using a learning process that repeatedly adjusts individual key centroids to move closer to the points user fingers actually touch when striking the keys. In this way, the key positions could potentially evolve to some configuration presumably closer to the fingers' preferred position.
U.S. Patent Application Publication No. 1997/5660488 also splits and angles the keyboard. However, it is different from the prior systems because it spreads out the angled key halves to fill the entire available entry area.
German Patent Application No. DE 10330002 A1 does not split the keyboard, but an interactive process allows portions of the keyboard to be independently shaped and angled (“morphed”) according to user touches.
All of the above systems are intended for two-handed use, and thus do not possess property (1). All of the above systems also angle the keys to match user hands (property (2a)); some adjust key placement to better fit user fingers (properties (2c) and (4)). However, none explicitly organizes the keys in groups assigned to fingers (property 2c), so it is likely that the distances required to strike individual keys would remain somewhat large and also irregular. All the above keyboards except for DE 10330002 A1, which uses a different character layout, are quite similar in layout to the Standard Layout (property (3)).
Several keyboards arrange the keys in an arc centered around the hand. One of these is U.S. Patent Application Publication No. 2012/0075194, which supports one-thumb typing. The keys are arranged in tiered arcs around the thumb (located in the lower corner) so that each key can be reached by the thumb's normal range of motion. Obviously, this system does not support multi-finger typing, and thus may be quite slow. Furthermore, the distance required to type any one key would be considerable, which may degrade accuracy and speed.
Another system using arcs is the HansOn Keyboard (e.g., see magnamus.tumblr.com), a one-handed keyboard that arranges the keys in concentric arcs around the hand. Both the position and the angle of the arcs can be adjusted interactively. The arc design reduces the distance required for one hand to type an arbitrary key from that required on a tablet-size Standard Virtual Keyboard. However, this distance is still sizable, because the keys are not organized around individual fingers. The distances and directions needed to type individual keys are also quite varied and irregular.
European Patent Application No. EP 2474890 A1 organizes keys around the fingertips of a user's hand: keys are assigned to some finger and then placed in a circular area around the point at which the fingertip touches the touch surface. This scheme, which possesses the property (2c), may minimize the distance a finger must travel to type a key. However, this scheme may be presented only abstractly. In other words, the application offers no plan for how keys from the Standard layout would be specifically allocated and organized. For a number of reasons, this scheme may be impractical. Even if only the 35-odd keys from the Standard Virtual Keyboard were allocated to the five fingers, this would mean seven keys per finger would be placed in a very small area around that finger. These small key sizes would most likely lead to inaccurate typing in practice. The movements required to type the keys may also be irregular, which might lead to further inaccuracy. While the system includes a method of fitting the keyboard to a user's hand (property 4), it possesses almost no structural similarity to the Standard Layout (property 3), and thus may be difficult to learn. This keyboard system may use space inefficiently, making it an unlikely fit for a space-constrained device like a smartphone.
U.S. Patent Application Publication No. 2005/0122313 organizes keys in reference to a Home Row of key positions obtained from the points at which the user's hand touches the touch surface. In an interactive process, the system maps certain keys from the Standard Layout directly to the Home Row positions, and then tries to map the remaining keys to positions which preserve similarity to the Standard Layout but also reflect the chosen Home Row positions. This keyboard therefore may possess property (2c), as well as properties (3) and (4). However, because this system only imposes a loose organization of keys that can leave them quite far away from fingers, any improvements in typing speed are likely to be limited.
Two existing keyboard systems organize keys into a columnar layout that provides a method of minimizing finger typing distances and promoting regular movements. These systems both possess property (2d). However, neither system places each column directly under one specific finger, limiting the accuracy that can be achieved. Plus, neither is designed for one-handed typing (property 1).
The first system, U.S. Design Pat. No. D639,807, shows a design for a mechanical, not a virtual, keyboard, and the keyboard is two-handed, not one-handed. This system organizes keys into vertical columns, some of which appear to be typed by one finger (property 2d). Each column appears to be sized and angled in an effort to match the expected size and angle of its associated finger. However, in addition to being inapplicable to a virtual, one-handed typing environment, the system has a drawback. Multiple key columns are assigned to one finger, such that no column is located exactly under the region occupied by that finger. Therefore, typing keys in the columns requires significant lateral movement, somewhat defeating the original purpose of columnar organization.
U.S. Pat. No. 5,336,001A is another keyboard system emphasizing a columnar layout; it is a virtual keyboard, although not one-handed. This system shares the shortcomings of the prior system in that its key columns are not located entirely under the region occupied by one finger, such that fingers must traverse greater distances to type keys in columns. Additionally, since the system is for two-handed typing, it is not a successful solution for tablet and/or smartphone typing. To be useful for one-handed typing, the key columns typed by one hand would somehow need to contain all the most frequently typed characters from the Standard Layout. Since there are many such characters, fitting them onto a set of key columns typed by only one hand may not be an easy problem to solve.
Consequently, while many methods of typing on a Standard Virtual Keyboard exist, most are awkward, inaccurate, and/or slow. Accordingly, there exists a need in the art to overcome the deficiencies and limitations described hereinabove.